Signature recognition system and method

ABSTRACT

A method of authenticating a signature including the steps of sampling a signature and storing data representative of the signature, converting the data to high dimensions vectors, feeding the high dimension vectors to an unsupervised neural network, performing a high order principal component extraction process on the high dimensions vectors to thereby identifying clusters of high dimension points, and analyzing the clusters of high dimension points to determine, based on previously stored information, the authenticity of the signature. Also an apparatus for such authentication including a sampling device for sampling a signature and storing data representative of the signature, a converting device connected downstream of the sampling device for converting the data to high dimension vectors, an unsupervised neural network for receiving the high dimension and performing a high order principal component extraction process on the high dimensions vectors to thereby identify clusters of high dimension points, and an analyzing device connected to the unsupervised neural network for analyzing the clusters of high dimension points to determine the authenticity of the signature.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a divisional of U.S. patent application Ser.09/482,075 filed Jan. 13, 2000, now U.S. Pat. No. 6,661,908, andentitled “Signature Recognition System and Method,” which claims benefitof U.S. Provisional Application Ser. No. 60/115,867 filed Jan. 13, 1999.

TECHNICAL FIELD OF THE INVENTION

The present invention is directed to signature recognition andauthentication and, more particularly, to a signature recognition andauthentication scheme employing unsupervised neural networks acting onvectors in high dimensional space.

BACKGROUND OF THE INVENTION

Various types of transactions require a party's signature as anindication of acquiescence to that transaction. For example, signaturesare necessary for checks, credit cards and numerous types of legaldocuments. As a signature often is the only necessary indication ofacquiescence to a transaction, forgery of signatures is of greatconcern.

Early anti-forgery schemes required comparison by a person of anoriginal signature kept on file and a newly executed signature on one ofthe aforementioned documents. Of course, such human intervention isterribly time consuming and often not reliable.

With increasing computing power, electronic signature recognition andauthentication systems have been developed. Such systems typicallyinclude an input device such as a digitizing pad or tablet to captureand digitally store the signature image and thereafter act on thatstored image in various ways to compare the new signature to apreviously-stored “authentic” signature.

For example, U.S. Pat. No. 5,745,598 to Shaw et al. discloses a methodwhereby a discrete cosine transform or orthogonal transform of thestored signature image is executed. A sequence of global parameters isgenerated and the image is divided into a plurality of strokes accordingto segmentation parameters based on the properties of the discretecosine transform or orthogonal transform. A sequence of featuremeasurements also are generated and, thereafter, the global parameters,segmentation parameters and feature measurements are stored asrepresentative of the signature. Comparisons are made based on thestored representative characteristics. The method disclosed by Shaw etal., however, is intended to be particularly useful for storing alimited amount of data on, for example, a magnetic card such thatverification of signatures can be accomplished at autonomous sites, suchas automatic teller machines. Because of the reduced amount of datacharacterizing any signature, there is, by definition, less reliabilityin verification.

In U.S. Pat. No. 5,559,895 to Lee et al., there is disclosed a writingpad with a graphics digitizer that converts the continuous lines of thesignature into digitized dots. The digitized dots are then located withrespect to a coordinate system, and horizontal and vertical coordinatesare assigned to each dot. The dots are also assigned values with respectto time. The resulting data represent the simultaneous accumulation ofboth static and dynamic information. These data are used to calculateeach feature of a set of features characterizing the signature. Thedatabase used to compare the current signature for the signatory (theperson making the signature) consists of a mean and a standard deviationfor each feature of the set. While such a system is an improvement overknown electronic signature authentication/verification systems, thissystem is focused on the multi-terminal transaction problem and it toolacks, the reliability necessary for superior signature authenticationand verification.

U.S. Pat. No. 5,812,698 to Platt et al. discloses a handwritingrecognition system that includes a preprocessing apparatus that usesfuzzy functions to describe the points of a stroke. The finalidentification of each character is performed by a neural network whichoperate on “sparse data structures” to identify the character'sfeatures. The Platt et al. system is directed to overall handwritingrecognition, not signature recognition per se, and thus is deficient inthe reliability of recognizing and/or authenticating a signature.

Other systems for signature verification has also been devised in theprior art as well. For instance, U.S. Pat. No. 5,442,715 to Gaborski etal. discloses a method and apparatus for cursive script recognition inwhich a digital signature is processed neural networks in a time seriesusing moving windows and segmentation. U.S. Pat. No. 5,465,308 toHutcheson et al. discloses a pattern recognition system where a twodimensional pattern is translated via Fourier transform into a powerspectrum and the leading elements of this power spectrum are then usedas a features vector and analyzed using a four layer neural network.U.S. Pat. No. 5,553,156 to Obata et al. discloses a complex signaturerecognition apparatus which utilizes stroke oriented preprocessing and afuzzy neural network to recognize and verify signatures. U.S. Pat. No.5,680,470 to Moussa et al. discloses a signature verification system andmethod in which a signature is preprocessed for test features which maybe compared against template signatures to verify the presence orabsence of the test features using conventional statistical tools. U.S.Pat. No. 5,828,772 to Kashi et al. discloses a method and apparatus forparametric signature verification using global features and strokedirection codes where the signature is decomposed into spatiallyoriented, time-ordered line segments. U.S. Pat. No. 5,825,906 to Obataet al. discloses a signature recognition system including apreprocessing subsystem which extracts feature vectors, a recognitionnetwork which recognizes patterns and a genetic algorithm used to decidewhich features are worth considering.

Other related technologies include Optical Character Recognition (OCR)systems and hardware for use in verification systems. For instance, U.S.Pat. No. 5,742,702 to Oki discloses a neural network for characterrecognition and verification which translates characters into a matrixand identifies the characters using a neural network. U.S. Pat. No.5,774,571 to Marshall discloses a writing instrument with multiplesensors for biometric verification which includes pressure sensitivecells.

However, these prior art systems fail to provide an effective andparticularly reliable signature authentication/verification system whichmay be readily commercially implemented. Furthermore, with theincreasing use of the Internet for a myriad of applications andtransactions, verifying accurately and reliably a signature on-line isparticularly desirable.

SUMMARY OF THE INVENTION

In view of the desire to provide an effective and particularly reliablesignature authentication/verification system, it is an object of thepresent invention to provide a signature authentication/verificationmethod and apparatus that preferably employs self organized neuralnetworks.

It is a further object of the present invention to minimize calculationtime and computer memory resources preferably by implementing apredefined process portion that implements hierarchical iconic zoomingto convert signature raw data. In an alternative embodiment, a“What/Where” network preferably replaces the hierarchical iconic zoomingprocess.

It is yet another object of the present invention to implement anunsupervised neural network to analyze the output of the hierarchicaliconic zooming stage. It is still another object of the presentinvention to provide at least one stage of component integration whereinthe response of the neural network is analyzed.

It is another object of the present invention to implement an improvedPi neuron in a second stage of component integration whereby an improvedresponse analysis can be performed.

It is still another object of the present invention to implement in asignature authentication system a means for assessing overgeneralizationand effectively counteracting the effects thereof.

In accordance with the present invention there is provided a signatureverification system that implements a unique combination of concepts toachieve the desired verification and authentication analyses. Oneconcept is recursive zooming, which is a process that takes signaturedata and converts the same to a set of vectors in high dimensionalspace. Another concept is execution of a cumulative ortho-normalizationprocess, a new method for calculating correlation ellipsoids or spheresthat contain a group of points in high dimensional space. While manyother concepts are described and combined to achieve the presentinvention, the two concepts mentioned immediately above, either alone orin combination with the other inventive features described herein have,to date, never been applied to a signature verification orauthentication system.

As discussed previously, the present invention is used to (1) verifyand/or authenticate a user's signature against forgery and/or to (2)biometrically recognize and/or verify a particular person. The methodand apparatus (system) of the present invention operates in two phases.In a first, or learning, phase, the system learns to recognize a user'ssignature. For this phase, the user provides several repeatable samplesof his signature. The system then analyzes the samples, identifiessignificant characteristics thereof and learns both to recognize thesignature itself and to distinguish the way it is written. In a second,or user verification, phase, the system determines if an input signaturematches the samples obtained during the first, or learning, phase.

Thus, in accordance with the present invention, it is significantly moredifficult to successfully forge a signature since the forger not onlymust know how a signature looks, but also how the signature is written.Consequently, the system of the present invention also is very useful asa biometric authentication device and method.

Generally, there are five main sub-systems comprising the presentinvention: input, recursive zooming, unsupervised neural network andcomponents integrator. Each of these is discussed in brief below and iselaborated upon in the Detailed Description.

(A) Input. The input component receives a signature using an inputdevice, e.g. a mouse, pen or tablet, and generates a description of thesignature. The description of the signature preferably is a listing oftime and corresponding location in x-y coordinates of the input device.

(B) Recursive zooming. The recursive zooming feature serves a pluralityof purposes. The first preferably is to convert the signature to astandard form. This is desirable in that several signatures by the sameperson are almost never identical. For example, the signature may besmaller or larger, stretched or slightly rotated. To be able torecognize any of these “same” signatures it is desirable that the systemignore such discrepancies. By converting the signature to a format thatdoes not depend on the signature size or rotation, the system can ignorethese factors and therefore can more accurately compare signatures.

Another feature derived from recursive zooming is conversion of thesignature to a form that easily can be handled by a downstream neuralnetwork. Because unsupervised neural networks (implemented in thepresent invention) learn to recognize collections of vectors in highdimensional space, the present invention preferably represents thesignature in such a collection. That is, the recursive zooming featureof the present invention converts the time/location representation intoa collection of vectors in a high dimensional space.

(C) Unsupervised neural network. Unsupervised neural networks are acollection of neurons that can learn to identify clusters of vectors inspace, where each neuron identifies a cluster. The network preferablyoperates in at least two modes. In the learning mode the neurons learnto identify the clusters or a portion thereof, and in the response mode,each neuron responds to vectors that likely belong to the cluster itlearned to recognize. In one preferable embodiment, ellipsoid neuronsare used for recognizing ellipsoid clusters. In another preferredembodiment, bubble-shaped neurons are implemented for recognizingclusters that are circular.

(D & E) Component integrators, first and second stages. In the learningphase, the component integrators analyze the network response to thesample signature. In the verification stage, the component integratorscompare the network response to the signature with the data collectedduring the learning process. If a “strong” match exists, the signatureis deemed authentic. If not, the signature is deemed likely forged.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the presentinvention will become apparent from the following detailed descriptionwhich is to be read in conjunction with the accompanying drawings, inwhich:

FIG. 1 depicts in schematic form the signature verification system ofthe present invention.

FIG. 2 shows various types of pointing devices operable with the presentinvention.

FIG. 3 depicts two intervals taken with respect to a sampled signature,in accordance with the present invention.

FIG. 4 shows a recursive iteration in accordance with the presentinvention.

FIG. 5 illustrates a dedicated ellipsoid neuron in accordance with thepresent invention.

FIG. 6 shows main ellipsoid directions in accordance with the presentinvention.

FIG. 7 shows variances of projected distance along Ui, i=1, 2, . . . n,in accordance with the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The following provides a detailed description of the preferredembodiments of the present invention, beginning first with definitionsof terms that are used herein.

Definitions

Soft Computing—Soft computing is a method by which problems are solvedwhen an algorithm is not available or can't be defined.

Neural Networks—A Soft Computing system, which solves problems by usingadaptive local resolution nodes. Each such node has input lines (inhuman neural cell called “dendrites”) and one output line (in humanneural cell called—“axon”). The node learns to respond to inputpatterns, which are received in the input lines (“dendrites”).

Self-Organized Neural Networks—A neural network, which learns toidentify characteristics of input patterns and hidden correlationwithout external intervention.

Intrinsic Geometry—A mathematical theory that deals with measurements ofgeometric objects such that the measurements do not depend on anycoordinate system. Usually the values which are explored by DifferentialGeometry and by Einstein's General Theory of Relativity (RicciCurvature=R_(kj)g^(kj)) are Intrinsic Scalars. Intrinsic geometry can berepresented by Tensors or Spinors (Gauge Theory) but it can be alsorepresented by high order neurons, which use Tensors or Spinors. Thisalso has significant value in the development of a new GeneralRelativity theory based on Self-Organized Neural Networks.

Generalization—The ability to recognize patterns which, differ from thelearnt patterns but have common characteristics with the learnt ones.This is the most important merit of neural networks. It represents atype of data compression. It turns the neural networks into systems,which solve problems, which were not encountered before.

“What/Where” neural network—A neural network in which there are at leasttwo layers. Each layer is a neural network. The first layer is usuallymade of different sizes of receptive fields. These receptive fields canhave different shapes but usually one common shape is used, such as adisk (filled circle). The input dendrites within a receptive fieldbecome the input of the first layer. This layer is called the “What”layer. Usually the “What” layer is made of High Order neurons, whichform a Self-Organized Competitive Neural network. The “Where” networkscans the original or processed pattern via receptive fields (or InputMask—a predefined shape of connected pixels). It uses the “What” networkas a look up table and therefore different areas on the pattern areidentified by different “What” neurons. For generalization purposes theresolution of the “Where” network is less than the resolution of thepattern. The output of the “Where” network can become the input of thenext “What/Where” layers. This description is typical as a uniqueinterpretation of Infilight Soft Computing Ltd. and therefore may differfrom common definitions.

“Eigenvalues” of a BiLinear Form—the values of vectors V for which therepresentation matrix A of the Bilinear Form is diagonal.

“Eigenvectors” of a Bilinear Form—the base vectors in which the Bilinearrepresentation matrix A of the Bilinear Form is diagonal.

diagonalization—A process by which a Bilinear Form representation matrixA becomes diagonal.

Principal Components—These are the “Eigenvectors” of the correlationmatrix.

High Order Principal Components—Vectors by which High Order Tensors canbe partially diagonalized. Usually Tensors of third orders and above cannot be diagonalized (a Tensor A_(ijk) can not be presented as A_(kkk)=λand for i≠j or i≠k or j≠k, A_(ijk)=0).

Correlation Matrix—The matrix of random variables correlations for whichA_(ij) equals Ex_(i)x_(j)−Ex_(i)*Ex_(j).

Unbounded Growth—An unwanted phenomenon in which internal values of aneuron grow too much. This problem can cause a competitive neuralnetwork to become a one neuron network (because the other neurons neverwin/learn). This problem is also called degeneration.

Pi-neuron—A neuron in which instead of using summations of inputdendrites, a multiplication is used. This neuron is usually used as astatistical AND operator.

Temporal summation neuron—A neuron that performs an integral operator onthe input values. In our model we just use a first order neuron. Inbiology, such neurons exist in the Thalamus (an area in the mid-brain)and have important role in primitive biological functions. Importantresearches about pain and Temporal summation were done by Dr.Lautenbauscher in Germany. (Tonic Pain Evoked By Pulsating Heat:Temporal Summation Mechanisms, by Lautenbauscher, Roscher, Strian;Somatosensory and Motor Research Vol. 12(1) pp. 59-75 (1995)).

Oja—Please refer to “Adaptive Cooperative Systems” by Martin Beckermanpp. 319-8.11.5,320-8.11.6 (1997, John Wiley & Sons, Inc., ISBN0-471-01287-4).

Linsker—Please refer to “Adaptive Cooperative Systems” by MartinBeckerman pp. 319-8.11.5, 320-8.11.6 (1997, John Wiley & Sons, Inc.,ISBN 0-471-01287-4).

Component Integration—A process by which a neural network verifies thatdifferent parts of a whole pattern exist simultaneously.

Next, the system of the present invention is described in detail.

Description of the Preferred Embodiments

The signature authentication/verification system of the presentinvention introduces a new concept for using an unsupervised neuralnetwork for signature authentication/verification. The inventioncomprises an adaptive cooperative system, which uses cooperation betweendifferent Unsupervised Neural Networks. The main process is divided intofive stages. Each stage performs a main sub-process, shown in FIG. 1. Asshown in the illustrated example of FIG. 1, the main process is dividedinto five stages or five sub-processes as follows:

A—Signature sampling—The system samples a signature as depicted atreference numeral 100, FIG. 1. Signature sampling is implemented via aninput device that translates hand movements to locations. The mostcommon devices are a mouse, mouse like devices, pressure sensitive pads,tablets, styluses and/or electronic pens. The signature samplingsub-system collects the device data and generates a list oftime/location pairs.

It is noted that some input devices can also sense pressure. Though thisinformation can be used for authentication, as different persons applydifferent pressure in the same place in a signature, a pressure variableis not relied upon by the present invention for reliable results sincemany input devices do not support pressure measurements. On the otherhand, the system of the present invention can be even further improvedby the addition of a pressure measurement.

Preferably, the input device provides information in sufficient detailto support the system of the present invention. Specifically, apreferable rate of information is at least about 40 samples per second.With lower sampling rates there typically is insufficient information tocharacterize a signature. Furthermore, the signature sampling processshould take, on the order of, about one second or more.

B—Predefined Process—The system translates the raw data into highdimension vectors, FIG. 1, element 200. These vectors represent thebiometric and geometric characteristics of the learned signature. Thevectors represent a unique “What/Where” analysis, which differs from theconcurrent Cooperating “What/Where” neural networks analysis. The methodtends to extract Intrinsic Geometry correlations.

More specifically, in the predefined process a recursive zooming processis implemented whereby time/location pairs generated by the signaturesampling are reconstituted into a set of high dimensional vectors. Allthe results of the recursive zooming are relative to the samplesignature. By using relative measurements, dependence on size androtation is avoided.

Each high dimensional vector is generated by recursively focusing onsmaller and smaller details of the signature. Each step in the processlooks at smaller details (defined by a shorter time span) and generatesmore coordinates in the vector. Various schemes can be devised for boththe zooming in and coordinates generation. However, in a preferredembodiment of the present invention, the predefined process preferablygenerates 26 dimensional vectors using 13 iterations. Vectors aregenerated as follows:

(a) The time period examined in the first iteration is between 50% and70% of the signature time (with steps of 2%);

(b) The time period examined in the second to 13th iterations is 70% ofthe time period examined in the previous iteration; and

(c) The time period in each iteration is either in the start of the timeperiod of the previous iteration, or in its end (but not in the middle).

Each iteration adds two coordinates to the vector. These coordinates arecalculated from the difference along the X and Y axes between inputdevice position at the start of the examined time period, and itsposition at its end. To avoid dependence on the size of the signature,the differences in locations are divided by the distance between theinput device locations at the start and the end of the time periodexamined in the previous iteration. In addition, there preferably isprovided a division-by-zero prevention algorithm. It should be notedthat the above described percentage of time period examined and thenumber of iterations is merely an example of one embodiment of thepresent invention. Theoretically, higher percentages (of time periodexamined) and higher iterations may be used which will yield betterresults. However, such increase in percentages and iterations wouldrequire increased computing and system capacity. Thus, under presentstate of technology and economics, the above embodiment has been foundto provide sufficiently accurate results considering the economic costs.

C—Unsupervised High Order principal components extraction—The systemlearns the clusters, at reference numeral 300, which are formed by thethus generated vectors in a high dimension Real Numbers Space—R^(n). Thesystem uses principal components elliptic extraction in a unique method.The principal components (“Eigenvectors” and “Eigenvalues” of thecorrelation matrix) are calculated using a geometric method ofCumulative Ortho-Normalization. This method saves the use of acorrelation matrix and it's diagonalization. Moreover, the vectors ofthe principal ellipsoid main directions may not need double precision 8bytes variables. Only the “Eigenvalues” require double variable types.The problem of unbounded growth of the ellipsoids is solved by a digitalcondition rather than using solutions such as Oja's solution orLinsker's model. This digital condition eliminates the use of punishmentrules on “too big” neurons.

In other words, in FIG. 1, neurons 310 in unsupervised neural networks300 learn to recognize clusters of vectors in a high dimensional space.The generic learning scheme is that the vectors are fed into thenetwork. The neuron closest to the added vector adds it to the clusterit recognizes, and modifies the location and size of the cluster toreflect the information gained from adding the new vector. The systemuses a “standard” neural network, but with two variations. The firstvariation is an unbounded growth rule, which is used to prevent oneneuron from growing too much so as to recognize all vectors as belongingto a single cluster. This rule is explained in more detail later herein.

The second variation is a unique process that is used for finding theresultant ellipsoids. When using ellipsoid neurons, each neuron learnsto identify an ellipsoid-shaped cluster. The problem is that the neuronhas to find the ellipsoid's main directions and sizes.

The standard approach to finding the ellipsoid main directions is tocalculate the correlation matrix of the distributions of the vectorsalong each axis. The Eigen-vectors of this matrix are the ellipsoid maindirections. The Eigen-values of the matrix are the variance of thevectors along each of the main directions. When taking the ellipsoidsizes to be twice the standard deviation along each of the maindirections, the ellipsoid covers 95% of the vectors, thus defining thecluster.

The major drawback of this approach is that whenever a new vector isentered to the network, the correlation matrix should be updated, andthe Eigen-vectors should be found. For large matrices, however, thisprocess is very time consuming.

Thus, in accordance with the present invention, there is provided amethod called Cumulative Ortho-Normalization which is a unique method tofind the main directions and sizes of ellipsoids and/or spheres. Themethod is based on the observation that the average of all the points inone hemisphere of the ellipsoid points to the main direction. To accountfor all the vectors in the cluster, the Cumulative Ortho-Normalizationmethod computes the average of all the vectors that are in onehemisphere and the reverse of the vectors in the other hemisphere. Tofind the other main directions of the ellipsoid, the process isrepeated, but each time, the components that lie along the maindirections found so far are subtracted from the vectors. Sizes are,again, computed as twice the standard deviation of the vector componentsalong the main directions.

To avoid the need to average all the vectors whenever a new vector isadded to the network, the system assumes that the new point does notchange the average by too much, thus, the system can rely on thedirections and sizes found after the previous vector was added. Theimplication of this assumption is that the ellipsoid found is anapproximation of the right ellipsoid. This approximation becomes betteras the number of vectors increases. Several hundred vectors result in anapproximation that is sufficient for all practical purposes.

It should be noted that analysis of the clusters created by therecursive zooming sub-system shows that these clusters tend to becircular. As such, ellipsoid neurons have little advantage overcircular/bubble-shaped neurons. Indeed, it has been found that anadvantage of bubble-shaped neurons is that they are symmetrical and as aresult, there is no need to find main directions, thus improving thelearning time and reducing the amount of memory necessary to implementthe system. Further, the radius of the balls is the standard deviationof the distance of the vectors for which a neuron won. Thus, suchcircular/bubble shaped neurons may be alternatively used in otherembodiments of the present invention. The radius of the balls preferablyis used only in the Component Integration—first stage process. This ispossible due to the special nature of the clusters that are formed bythe predefined process, Le., recursive zooming. However, becauseellipsoid neurons provide improved accuracy (although the improvementmay be small), the embodiment using ellipsoid neurons are discussed infurther detail. In this regard, it should be appreciated by a personskilled in the art that the teachings regarding the ellipsoid neuronsmay be readily applied to circular/bubble shaped neurons. In deed, thecircular/bubble shaped neurons may be considered to be a specialembodiment of the ellipsoid neurons discussed herein.

D—Component Integration—first stage—Component integration 400 is amethod for verifying that the geometric and biometric components of thelearnt signature also exist in a compared signature.

This method relies on three types of unique neurons as follows:

D.1—The Temporal Summation neuron, 410

D.2—The Average Temporal Summation neuron, 420

D.3—Pi-neuron, 430

D.1) In the Component Integration—first stage, each temporal summationneuron 410 learns the relation between the number of vectors within anellipsoid and the total number of vectors. It remembers the average ofthese ratios and the standard deviation.

D.2) A second type of neuron, the average temporal summation neuron 420,learns the average distance of vectors within an ellipsoid. It learnsthe average of these averages and their standard deviation.

D.3) The Pi-neuron 430 multiplies the statistical distance of the lasttwo neurons.

E—Component Integration—second stage—This neuron is an improvedPi-neuron 500 wherein, unlike a regular Pi-neuron that multipliesunprocessed values, it sorts its input dendrites in a descending orderof intensities and finds the minimal sorted index for which themultiplication is less than one. The maximal worst case happens whenthis index has its greatest value. This means that too many TemporalSummation and Average Temporal Summation neurons report big standarddeviations. Non-Dominant Characteristic Extractor means 510 is provided,which extracts the most deviated by multiplying the sorted values ofstandard deviation or variances.

As a result of this analysis, the system calculates the abnormaldeviation from the compared/learnt original signature.

In a more generalized explanation, the purpose of the componentintegrator is to match the neural network response to a signature withthe neural network response to the samples provided in the learningphase. For each signature, and for each neuron, the neuron response tothe signature is calculated. The neuron response to a signature isrepresented using two numbers. The first number, which is named the“global response,” is the percentage of the vectors the neuron respondedto. The second number, the “local response,” is the average of thedistances from a vector the neuron responded to, and the center of theneuron.

The component integrator measures by how much the response of a neuronto a signature deviates from the average response to the samplesignatures. These numbers are then fed to an improved Pi neuron 500,whose result is the measure of matching between the signature and thesamples provided in the learning phase. Improved Pi neurons arecomponents that operate in two modes. In a learning mode they learn thenumber of significant inputs they have and in an operation mode, theyoutput the multiplication of the significant inputs. A more detailedexplanation of this aspect of the present invention follows.

It is again noted that the above discusses merely one example of thepresent invention. The remainder of this disclosure is directed to acomprehensive mathematical discussion of each of the above sub-systemsor components in accordance with one embodiment of the present inventionand, ultimately, how the entire system functions as a whole unit,including the differences between learning and comparison.

Mathematical and Comprehensive Description

Again, the signature authentication system of the present inventioncomprises five sub-processes or sub-systems as follows:

A—Signature sampling, 100:

B—Predefined process—Biometric & Geometric Analysis, 200.

C—High Order principal components extraction process, 300.

D—Component Integration—first stage process, 400.

E—Component Integration—second stage process, 500.

A—Signature Sampling Process

Sampling in the present invention preferably is implemented with apersonal computer 20 with any attached standard input pointing device,like devices 21, 22, 23, 24 shown in FIG. 2, or any other like device.An application running on personal computer 20 samples the humansignature in real time (by using a high priority thread) at asufficiently high constant rate, preferably approximately every 7milliseconds. In a preferred embodiment, the signature sampling processis initiated only when a click event occurs. The followingthree-dimensional values are sampled:

-   -   X—x coordinate.    -   Y—y coordinate.    -   Δt—derived from the constant sampling rate.        These three-dimension vectors are buffered into a raw data        array. Signatures can differ in the length of their raw data        array. Incidentally, the click value may also be relied upon as        an additional feature by calculating the distance between two        discontinued points. Furthermore, and as mentioned previously,        because many input pointing devices can not sample the pressure,        pressure need not be monitored in a best mode of the present        invention. However, using an appropriate device for sampling        values of the pressure, a fourth dimension can be defined. And,        an even more accurate signature authentication can be achieved        by using this additional dimension.        B—Predefined Process—Biometric & Geometric Analysis.

In this process the purpose is to convert the three dimensions raw datavectors to high dimensions vectors. In that specific application theconversion process translates the three dimensions vectors into 26dimensions vectors. The conversion vectors are generated, by ahierarchical zooming on the time intervals of the signature. Forexample, a sampled signature consists of 201 points, 200 time intervalsof 7 milliseconds each. For such a signature, an array of 201two-dimensional points is built. The time is included in the array indexbeginning from 0 up to 200.

The first hierarchical zooming will be on the time interval from t₀ tot_(max). The system uses several segmentation rates as follows. Supposethat the segmentation rate equals to 0.7 and two intervals are chosen;one interval is between 0 milliseconds to 140 milliseconds and the otheris between 60 milliseconds to 200 milliseconds, as shown in FIG. 3.These two time segments represent a single iteration. The ruleimplemented in the present invention for the segmentation preferably isas follows.

First Interval Indices are:

-   -   Interval _(I=1,a)=(start index, start index+(end index−start        index)×0.7)    -   Interval _(I=1,b)=(end index−(end index−start index)×0.7, end        index)

The segmentation process is a recursive process which repeats on eachone of the parent intervals for 13 zooming iterations. This process canbe thought of as a “What/Where” iconic conversion. Every iterationcalculation is based on a single rate. A recursive iteration of theright branch is shown in FIG. 4.

For stability reasons, the process is repeated for differentsegmentation rates, once for 0.5 (i.e., no overlapping between the twointervals), the next for 0.52, the next for 0.54, and so on up to rateof 0.7. Each iteration records the two dimensional vectors, which aregenerated by subtracting the start point from the end point. Thedifference vectors are being divided by the length of a parent iterationvector (equipped with a “division by zero” prevention algorithm) inorder to detect internal signature size invariant proportions. Thisprocess explores proportional geometric and biometric relations.

In a preferred embodiment of the present invention, the zooming ratio isdefined as the 13th root of 2/n where n is the number of distinct (x,y)points. This embodiment ensures that after 13 iterations, the timeinterval is between 2 successive samples. For example, assume thesignature consists of 200 (x,y) points, then ( 2/200) to the power of1/13 is about 0.7. This means that the intervals shrink by a factor of0.7 in each iteration. The instant embodiment is particularly desirablewith relatively long signatures. Specifically, in such longersignatures, the last iteration, without the implementation of theinstant scheme, would reach final intervals that are too long, wherebythe system becomes insufficiently sensitive to local features of longsignature curves.

In accordance with the preferred embodiment of the present invention,however, the system truncates long signatures. This does not inhibit theverification process because in long signatures there are enoughfeatures such that instability at the end of the signature (due tounstable truncation) is compensated for. Accordingly, where othersignature verification systems might implement more neurons, the presentinvention, with a limited number of neurons, achieves sufficientreliability based, possibly, only on a hand gesture resulting in threeto five Latin letters.

It is noted that in the present embodiment of the present invention, thenumber of neurons used is 32 and the dimension of the vector space whichis analyzed by the neural network is 26. Each iteration of the recursivezooming generates a two dimension vector and 13 iterations are executedresulting in 13×2=26 dimensions. However, in alternative embodiments,different numbers of neurons and dimensions of vector space as well as adifferent number of iterations may also be used depending on the desiredaccuracy and the system capacity. Thus, the example discussed hereinshould not be considered as a limitation of the present invention butmerely one example.

The above process converts the three dimensions raw data vectors, intohigh dimensional vectors of 26 dimensions. The 26 dimensions are derivedfrom the collections of 13 recursive intervals (zooming in), eachinterval's end—start vector has x & y coordinates and these coordinatespreferably are recorded into a buffer. The buffer becomes full each timethe recursive process reaches the maximal depth 13. Each iteration fillstwo places in the buffer. In a preferred embodiment, when the buffer isfull, i.e., has 26 values, it is written into a sequential records file.The entire recursive process preferably lasts only a few seconds.

This process will not always generate the same vectors; however, thewhole set of vectors can be represented as the unification of clustersof 26 dimensional points. Points are not just sporadically scattered inR²⁶. The fact that the clusters will not always repeat, is well treatedby the “improved Pi neuron” which is the last processing phase of theneural network, discussed later. Indeed, the clusters' irrepeatabilitywas the incentive for the improved Pi-neuron. The output file of thepredefined process becomes the input of the neural network 300.

In a human analogy, the predefined process acts like the early visionstages in the visual cortex of the vertebrates.

C—High Order Principal Component Extraction Process.

Principal component extraction is executed in an unsupervised neuralnetwork 300 by a method which implements geometry to extract the mainellipsoid directions. Such neurons are known as Second Order Neurons.The neuron learns to identify clusters of high dimension points using ahigh dimension ellipsoid. Each neuron uses a dedicated ellipsoid. As anexample, such an ellipsoid is shown in FIG. 5. In this Figure it isreadily seen that the described ellipsoid has two main directions(according to a flat two dimensions cluster of points).

In the unsupervised neural network of the present invention, the neuronidentifies close points, which form a cluster and gradually “moves” it'scenter point at a predefined rate to the middle of the cluster (themiddle of the cluster is calculated by averaging it's points). This isdone by competitive self-organization. The main directions of theellipsoid are gradually updated with each new point that the neuronlearns. This unique technique is described in the following paragraph.

The ellipsoid main directions extraction is accomplished by the unique“Cumulative Ortho-Normalization” technique.

Let V denote the difference vector between the position value of theneuron (center point of the ellipsoid) and a new learned value. Let Pdenote the position of the neuron in n dimensions (center point of theellipsoid). Let U₁, U₂, . . . U_(n) denote the main ellipsoid unitdirections. And, let λ₁, λ₂, . . . , λ_(n) denote the projectionvariances of the cluster points on the main ellipsoid directions. Eachof these defined values is depicted in FIG. 6.

It is important to note that the 2*√λ calculated value (where λrepresents the variance) counts for most (about 95%) of the sample spacewhich is included in that range.

Assume that the A values are the Eigenvalues of the correlation matrix.To refrain from using a conventional high time complexity correlationmatrix and its diagonalization, the following new technique is employedwhereby main directions extraction is simplified.

The mathematical values of the λ's are the variances of the projecteddistance along the U_(i), i=1,2, . . . n unit directions. An example fori=2 is shown in FIG. 7. In the following section and associated figures,m will denote the number of learned vectors (samples).

The U_(i)s are set to initial values of a multiplication of the ordinarybase in order to avoid too small ellipsoids. For example, U1=(100,0,0,0,. . . ), U₂=(0,10,0,0, . . . ), U₃=(0,0,100,0, . . . ), . . . etc.

The updating process starts from i=1 up to i=n, (in 26 dimensions n=26).

U₁ will serve as the highest main ellipsoid's direction; U₂ will serveas the second ellipsoid's direction, etc.

The term: [V*<V, U₁>] is added

To: [m*λ₁*U₁] where m is the number of learnt vectors. (The old m*λ₁*U₁is actually the square ellipsoid main direction).

Let the new main square direction be denoted vector by Y, whereY=V*<V, U ₁ >+m*λ ₁ *U ₁.

Another way to define Y is:Y=V*Sign(<V, U ₁ >*∥V∥+m*λ ₁ *U ₁, where ∥V∥ is the norm of V.

If the inner product <V, U₁>, which is a projection on VI is negative,then the added vector V*<V, U₁> points to the positive hemispheredirection which is pointed by U₁. The positive hemisphere is all the Zvectors such that <Z, U₁>>=0.

If <V, U₁> is positive then the V*<V, U₁> vector is also in the positivehemisphere which is formed by U₁. This means that summations will notcancel each other but will rather work in a cumulative manner. This isthe basis for the Cumulative Ortho-Normalization technique.

The new U₁ vector will be U₁=Y/|Y|. Therefore, it will be a new unitvector.

The new λ₁ is just calculated as: (m*λ₁+<V,Y/|Y|>²)/(m+1), where m isthe number of previous points that the neuron has learned. This meansthat a new squared sum is added to the old sum of square projections onthe VI direction. So m*λ₁ is actually the sum of all squares.

λ₁=Sum of square projections on the first main direction divided by thenumber of learned vectors (samples). The new m*λ₁*U₁ vector is the newsquare main ellipsoid direction.

The process then proceeds to U₂ . . . .

We add V*<V, U₂> to m*λ₂* U₂.

Again we have Y=V*<V, U₂>+m*λ₂*U₂.

Or, in another way, V*Sign (<V, U₂))*∥V∥+m*λ₂*U₂.

It is then preferable to keep Y perpendicular to the already calculatedU₁.

Accordingly, a new vector Z=Y−<Y, U₁>*U₁ is calculated.

By this the component of Y which is parallel to U₁ is subtracted.

Z is then normalized and a new U₂=Z/|Z|.

Again, the new λ₂ will just be calculated as (m*λ₂+<V, Z/|Z|>²)/(m+1).In that way the square value of the projection on the new V2 is added tom*λ₂ and divided by m+1 so λ₂ will just be the sum of squaredprojections divided by the number of learned vectors (samples).

The process continues by adding V*<V, U₃> to m*λ₃*U₃.

Again we have Y=V*<V, U₃> to m*λ₃*U₃.

Or, in another way V*Sign (<V, U₃))*∥V∥+m*λ₃*U₃.

The projections are then subtracted on the previous main directions so:Z=Y−<Y, U ₁ >*U ₁ −<Y, U ₂ >* U ₂

The new U₃ unit vector will just be new U₃=Z/|Z|.

Again, the new λ₃ will just be calculated as (m*λ₃+<V, Z/|Z|>²)/(m+1).

This process continues until all the main ellipsoid directions areupdated. After the last main direction is calculated, the processapproaches its end.

Additionally, a variable P is also kept. P is the average of the learnedvectors and also an additional point, which is used as the neuronlocation. This may sound somewhat unusual, but evidently when using Pfor ellipsoid calculations and L (neuron location) for competition, thenetwork differentiation is improved. The process also uses a rate bywhich L approaches P. According to the present invention, L does notmove towards a new learned point but rather towards P, the average ofall the learned points. While this duality might use additional memory,the advantages obtained thereby outweigh any apparent disadvantages.

In the preferred embodiment, there is provided in the unsupervisedneural network 300 unbounded growth prevention rules. By these rulesthere is eliminated the situation of one dominant winner ellipsoid (thatlearns too much data) or, on the other hand, becomes a degeneratedellipsoids. Each ellipsoid has a winning counter. Whenever the neuronwins, the counter is incremented. The Neural Network becomes lesscompetitive if a winning neuron has over two times winnings over a closeneighbor. At that stage the neighbor will be also updated for inputvectors with distances which fulfill the condition,

(Minimum Distance from Winning Neuron)/(Distance from Neighbor)>=0.7.

Whenever a neuron wins “too many times”, the network behaves like aKohonen network. The condition acts as a fortification of weak neuronsand therefore acts as a constructive condition. It is important to saythat it is a good condition for connected clusters but will not alwaysbe appropriate for data, which is scattered in R n in disconnectedclusters.

To summarize the above, in the present high order principal componentsextraction process the topology of the signature is learned by theunsupervised second order ellipsoid neural network. The learnedsignature topology includes biometric and geometric characteristics.

D—Component Integration—First Stage Process.

After the second order neurons (High Order principal components) finishlearning the topology of the data vectors in R²⁶, the componentintegration 400 starts.

All the sample signatures are sent to the system one by one again. Theprocess uses the signatures and builds the grounds for the verificationthat geometric and biometric characteristics are preserved in futurecompared signatures.

The second order ellipsoid neurons are kept fixed while two statisticmeasurements are learned for each signature and for each second orderellipsoid neuron as follows:

The process learns the proportion r between the number of vectors withinan ellipsoid and the total number of vectors. It learns the averagevalue of r and the standard deviation of r. r is calculated in relationto the number of sample signatures. This process of calculating r iscalled Temporal Summation.

A second process averages the distance of all the vectors within anellipsoid. Distances are measured from the center of the ellipsoid.Here, let s denote this value. The process learns the average value of sand it's standard deviation when testing all the sample signatures. s iscalculated in relation to the number of sample signatures. This processof calculating s is called Average Temporal Summation. These calculatedvalues are then passed to conventional Pi-neurons 430.

Thus, in summary, the accumulation of data which is performed by theTemporal Summation and the Average Temporal Summation is used as a meansfor extracting data out of the large numbers of vectors, not necessarilyout of any preferred cluster/bubble. Even if the neurons miss the centerof the cluster, the Temporal Summation and the Average TemporalSummation are useful.

E—Component Integration—Second Stage Process

All the sample signatures are then passed to the improved Pi-neuron 500one by one for further component integration. Referring to the lastparagraph of the previous section, two values are calculated for eachsignature and for each second order neuron.

A=(Average r−current signature sample r)²/(Variance of r).

B=(Average s−current signature sample S)²/(Variance of s).

The output value for each one of the ellipsoids is the multiplicationA*B, where

B—measures local structure deviations, and

A—measures global structure deviations.

The process treats these values as independent probabilities. This isone of the reasons for the multiplication. Another more basic idea isthat A*B is usually more stable than A+B or A or B. This can be shown inexperiment.

For each second order neuron the result A*B is an input of the lastneuron, the improved Pi-neuron 500. Again, it is a multiplicationneuron.

Let us denote the values of A*B of each ellipsoid as X_(i) such thati=1,2, . . . k, where k equals to the number of neurons. The X_(i)values are sorted in descending order where X₀ is the greatest value.The neuron starts multiplying the X_(i) values until the multiplicationis less than 1. If no such condition is achieved, the signature isdiscarded.

The first i which satisfies X₀*X₁* . . . *X_(i)<1 is remembered. Themaximal value of i is learned by the improved Pi-neuron 500. A value of1 or 2 is added to Maximum i. The new i+1 or i+2 value is denoted by J.

When the system compares a new signature, the X_(i) values are sortedand the multiplication up to index J is calculated. If the value of themultiplication is greater than 1, the system identifies the signature asa false one.

The ideas behind the Improved Pi Neuron according to the presentinvention include (1) the multiplication is a means of verifying thatsignature characteristics simultaneously appear (multiplication is likethe AND operator) and (2) it is preferable to avoid a multiplication oftoo many small values, which can cause false verifications.

The multiplication allows deviations of the characteristics of thetested signature as long as there are still enough characteristics,which are unique to the person who owns the signature.

Characteristics of the Improved Pi Neuron

The number of required X_(i), which is the number of second orderneurons, depends on the signature complexity. For example, forsignatures effected with a mouse, 20 neurons typically are sufficient.For long signatures, on the other hand, delivered via a digitizing pad,32 to 40 neurons are more preferable.

Assessment of Overgeneralization

In addition to the above-described five major components 100, 200, 300,400 and 500, the present invention preferably also implements a criteriafor assessing Overgeneralization. Overgeneralization is a situation inwhich the irrepeatability or over-simplicity of the learnt signature cancause the system to accept false signatures as authentic. In general, iflout of, for example, 7 letters of a signature is spoiled, then thesystem should still reliably identify the signature. This desirablesystem characteristic is made possible by the compensation process whichexists in the Improved Pi Neuron.

If too many changes in the original signature are still accepted, thenthe system is termed as Overgeneralized. The standard deviation, whichis learned by the Temporal Summation and Temporal Summation neurons,preferably should have small values in comparison with the averagevalues. To make sure that no overgeneralization exists, it is sufficientthat half of the Neurons of the Component Integration—First Stage 400agree with the condition:

-   -   Average*Average/Variance>50 (or other predefined threshold)

In a preferred embodiment, the system requires that at least 10(Temporal Summation & Average Temporal Summation) neurons (out of32*2=64) comply with such a condition. Stricter requirements can beimposed for at least 16 to 32 neurons out of the 64 of the ComponentIntegration—First Stage.

It is important to note that the system of the present inventionimplements the instant special tool for the assessment ofOvergeneralization, which tool is rarely, if ever, incorporated in othersignature authentication systems. Indeed, the implementation of the toolis a direct result of the structure of the Component Integrationprocess.

Criterion for Canceling Overgeneralized Neurons

As noted above, the signature verification system of the presentinvention preferably uses the following condition for cancelingovergeneralized Temporal Summation and Average Temporal Summationneurons:

-   -   Average*Average/Variance>Predefined Threshold.        That is, the variance preferably has to be relatively low. Table        1 presents statistical values of a neural network trained 10        times with different signatures. It is expected that the network        contains no information.

TABLE 1 Competitive Temporal Summation Average Temporal Summation Neuronindex Average{circumflex over ( )}2/Variance Average{circumflex over( )}2/Variance 0 6.4 2.8 1 7.7 2.2 2 3.3 1.7 3 1.3 0.6 4 0.1 0.1 5 2.51.1 6 0.8 0.7 7 3.0 0.8 8 6.8 2.3 9 1.8 0.9 10 0.1 0.1 11 0.6 0.4 12 0.10.1 13 0.1 0.1 14 0.1 0.1 15 0.1 0.1 16 5.9 1.3 17 6.9 2.4 18 2.9 0.9 190.1 0.1 20 0.8 0.8 21 0.1 0.1 22 0.1 0.1 23 7.4 1.2 24 0.6 0.6 25 0.20.2 26 0.1 0.1 27 0.1 0.1 28 0.6 0.5 29 3.3 2.0 30 3.2 1.6 31 0.8 0.6

On the other hand, comparison with Table 2 of values of a network, whichwas trained with 10 repeated signatures of the same person, shows theway significant statistical values are represented by the neuralnetwork. The analysis shows the statistical basis of the componentintegration implementation. It also shows the justification for thecriterion for the cancellation of overgeneralized neurons.

TABLE 2 Competitive Average Neuron Temporal Summation Temporal Summationindex Average{circumflex over ( )}2/Variance Average{circumflex over( )}2/Variance 0  78.16  15.36 1  169.02  9.54 2  864.07  18.83 3 325.66  25.53 4  204.73  10.68 5  281.87  12.78 6  71.39  71.55 7 733.43  28.84 8  46.89  34.66 9  536.31  72.46 10 2381.02 129.76 11Canceled (Weak Neuron) Canceled (Weak Neuron) 12 1011.12  55.8 13  71.95 9.57 14  932.93  36.26 15  516.32  76.62 16  95.08  5.24 17   0.426(Over  0.45 (Over generalized) generalized) 18  212.03  25.62 19  426.96 35.49 20  45.82  7.11 21  17.44  2.88 22  58.52  5.19 23  382.54 123.9424  787.41 127.36 25  199.71  33.97 26  286.72  31.08 27  83.22  30.2928  226.77  25.18 29  221.87  26.67 30  287.85  17.25 31  205.08  16.74

The system of the present invention has two modes, one for updating theneural network weights (the learning mode) and one for comparing a newsignature's features to the already learnt features (the comparingmode).

The Learning Mode—Phase 1.

-   -   Signature sampling—The system samples a signature.    -   Predefined Process—The system translates the raw data into high        dimension vectors.    -   Unsupervised High Order principal components extraction—The        system learns the clusters, which are formed by these vectors in        a high dimension Real Numbers Space—R^(n).        The Learning Mode—Phase 2.    -   Signature sampling—The system samples a signature.    -   Predefined Process—The system translates the raw data into high        dimension vectors.    -   Unsupervised High Order principal components comparison—The        process does not update the ellipsoid's main directions. It just        feeds the next layer.    -   Component Integration—first stage—the averages of s and r and        their variances are learned.        The Learning Mode—Phase 3.    -   Signature sampling—The system samples a signature.    -   Predefined Process—The system translates the raw data into high        dimension vectors.    -   Unsupervised High Order principal components comparison—The        process does not update the ellipsoid's main directions. It just        feeds the next layer.    -   Component Integration—first stage—the layer calculates A and B        and their multiplication    -   Component Integration—second stage—The Improved Pi Neuron learns        the index J        By the above three stages, the learning process is concluded.        The Comparison Mode

This mode has only one stage as follows:

-   -   Signature sampling—The system samples a signature    -   Predefined Process—The system translates the raw data into high        dimension vectors    -   Unsupervised High Order principal components comparison—The        process does not update the ellipsoid's main directions; it just        feeds the next layer    -   Component Integration—first stage—the layer calculates A and B        and their multiplication    -   Component Integration—second stage—The Improved Pi Neuron        calculates the multiplication of A_(j)*B_(j)*A₂*B₂* . . .        *A_(j)*B_(j) up to and including index J. The multiplication is        just one or two index numbers above the worst case maximal index

The result of the multiplication should be less than 1. If it is greaterthan 1, it means that the new compared signature differs from thelearned signatures. A result of between 1 and 100 is designated a grayrange which usually means that some similarity exists but certainly notenough to be sure that the signature is not a forgery. Of course, theactual scale or criteria implemented will depend on specificrequirements.

In view of all of the foregoing, it is readily seen that the presentsignature authentication system is different from prior art systems.Specifically, the early preprocessing is executed by an algorithm andnot by a neural network. This is done in order to reduce memory and diskspace requirements. Furthermore, the preprocess implements ahierarchical iconic zooming process in order to convert the signatureraw data coming from the signature sampling sub-process. In analternative embodiment, the preprocess is replaced by a “What/Where”network if no biometric data is available. In the preferred embodiment,the signature raw data is converted to 26 dimensions high order vectors.

There are also important improvements in the neural networks aspect ofthe present invention. Specifically, (1) the data storage in the systempreferably is solely in self organized neural networks. (2) In thesecond order neurons a geometric method of training/learning is used. Noexplicit diagonalization exists. There is no explicit use of acorrelation matrix. By this, calculations become much faster. This isvery important because self organized neural networks usually need manyiterations and therefore are time consuming. (3) The second orderneurons feed another layer of first order neurons. This concept iscontradictory to the model in which high order neurons are the last “busstop”. (4) Pi (π) neurons are used in the Components Integration phase.This is not very common in neural networks, and heretofor unknown insignature authentication systems employing neural networks. (5) The lastoutput Pi neuron that is used, is not an ordinary Pi neuron. It is aneuron which sorts the input values and then calculates themultiplication up to a “stability index”. (6) Finally, the presentinvention implements cooperative unsupervised neural networks. Incontrast, well-known prior art signature authentication solutionsusually adopt supervised neural networks.

It is noted that instant invention can be implemented fully either on apersonal computer or mainframe type computer, or any combinationthereof. Additionally, such computers preferably are connected inconventional manner to the Internet to facilitate on-line transactions.

While the present invention has been described with reference topreferred embodiments, it will be understood by those skilled in the artthat various changes may be made and equivalents may be substituted forelements thereof without departing from the scope of the invention. Forinstance, as previously noted, circular/bubble shaped neurons may beused instead of elliptical neurons. In addition, many modifications maybe made to adapt a particular situation or material to the teachings ofthe invention without departing from the essential scope thereof.Moreover, different numbers of dimensional vectors and differentiterations may be used than the above discussed embodiment to increaseaccuracy or as required by system requirements. Therefore, it isintended that the invention not be limited to the particular embodimentsdisclosed as the best mode contemplated for carrying out the invention,but that the invention will include all embodiments falling within thescope of the appended claims.

1. A method of electronically learning a signature, comprising the stepsof: sampling a signature and obtaining raw data representative thereofusing a recursive sampling process; translating the raw data into highdimension vectors; and extracting, via an unsupervised neural network,high order principal components of the high dimension vectors bycumulative ortho-normalization.
 2. The method of claim 1, furthercomprising integrating the high order principal components by generatinga value r corresponding to a ratio of the number of vectors within anellipsoid to the total number of vectors and a value s, the value scorresponding to the average of distances of all vectors within theellipsoid.
 3. The method of claim 2, further comprising: calculating avalue A=(average r−current signature sample r)²/(variance of r) andB=(average s−current signature sample s)²/(variance of s); andmultiplying the values A and B together.
 4. The method of claim 3,wherein multiplying the values A and B together comprises multiplyingthe values A and B together in a Pi neuron.
 5. Software forelectronically learning a signature, the software encoded in media andoperable when executed to: sample a signature and obtaining raw datarepresentative thereof using a recursive sampling process; translate theraw data into high dimension vectors; and extract, via an unsupervisedneural network, high order principal components of the high dimensionvectors by cumulative ortho-normalization.
 6. The software of claim 5,further operable to integrate the high order principal components bygenerating a value r corresponding to a ratio of the number of vectorswithin an ellipsoid to the total number of vectors and a value s, thevalue s corresponding to the average of distances of all vectors withinthe ellipsoid.
 7. The software of claim 6, further operable to:calculate a value A=(average r−current signature sample r)²/(variance ofr) and B=(average s−current signature sample s)²/(variance of s); andmultiply the values A and B together.
 8. The software of claim 7,wherein the software operable to multiply the values A and B togethercomprises the software operable to multiply the values A and B togetherin a Pi neuron.
 9. A computer for electronically learning a signature,comprising: memory; and one or more processors collectively operable to:sample a signature and obtaining raw data representative thereof using arecursive sampling process; translate the raw data into high dimensionvectors; and extract, via an unsupervised neural network, high orderprincipal components of the high dimension vectors by cumulativeortho-normalization.
 10. The computer of claim 9, the one or moreprocessors further operable to integrate the high order principalcomponents by generating a value r corresponding to a ratio of thenumber of vectors within an ellipsoid to the total number of vectors anda value s, the value s corresponding to the average of distances of allvectors within the ellipsoid.
 11. The computer of claim 10, the one ormore processors further operable to: calculate a value A=(averager−current signature sample r)²/(variance of r) and B=(average s−currentsignature sample s)²/(variance of 5); and multiply the values A and Btogether.
 12. The computer of claim 11, wherein the one or moreprocessors operable to multiply the values A and B together comprise theone or more processors operable to multiply the values A and B togetherin a Pi neuron.
 13. A system for electronically learning a signaturecomprising: means for sampling a signature and obtaining raw datarepresentative thereof using a recursive sampling process; means fortranslating the raw data into high dimension vectors; and means forextracting, via an unsupervised neural network, high order principalcomponents of the high dimension vectors by cumulativeortho-normalization.